<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "https://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en-US">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=11"/>
<meta name="generator" content="Doxygen 1.12.0"/>
<meta name="viewport" content="width=device-width, initial-scale=1"/>
<title>NeuZephyr: D:/Users/Mgepahmge/Documents/C Program/NeuZephyr/include/NeuZephyr/OperationKernels.cuh File Reference</title>
<link rel="icon" href="NZ_logo2.png" type="image/x-icon" />
<link href="tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="dynsections.js"></script>
<link href="navtree.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="resize.js"></script>
<link href="doxygen.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<table cellspacing="0" cellpadding="0">
 <tbody>
 <tr id="projectrow">
  <td id="projectlogo"><img alt="Logo" src="NZ_logo2.png"/></td>
  <td id="projectalign">
   <div id="projectname">NeuZephyr
   </div>
   <div id="projectbrief">Simple DL Framework</div>
  </td>
 </tr>
 </tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.12.0 -->
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:d3d9a9a6595521f9666a5e94cc830dab83b65699&amp;dn=expat.txt MIT */
$(function() { codefold.init(0); });
/* @license-end */
</script>
  <div id="navrow1" class="tabs">
    <ul class="tablist">
      <li><a href="index.html"><span>Main&#160;Page</span></a></li>
      <li><a href="pages.html"><span>Related&#160;Pages</span></a></li>
      <li><a href="namespaces.html"><span>Namespaces</span></a></li>
      <li><a href="annotated.html"><span>Classes</span></a></li>
      <li class="current"><a href="files.html"><span>Files</span></a></li>
    </ul>
  </div>
  <div id="navrow2" class="tabs2">
    <ul class="tablist">
      <li><a href="files.html"><span>File&#160;List</span></a></li>
    </ul>
  </div>
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:d3d9a9a6595521f9666a5e94cc830dab83b65699&amp;dn=expat.txt MIT */
$(function(){ initResizable(false); });
/* @license-end */
</script>
<div id="nav-path" class="navpath">
  <ul>
<li class="navelem"><a class="el" href="dir_d522931ffa1371640980b621734a4381.html">Users</a></li><li class="navelem"><a class="el" href="dir_a7e6ee1ae3f772c9504a0b543f2027e2.html">Mgepahmge</a></li><li class="navelem"><a class="el" href="dir_e03f57e346cc4845a4c354a35630b169.html">Documents</a></li><li class="navelem"><a class="el" href="dir_231a0482af2b83c895f27ba7fe745141.html">C Program</a></li><li class="navelem"><a class="el" href="dir_0fa7fc3a0dfd304dbfc9dce9f6facfa2.html">NeuZephyr</a></li><li class="navelem"><a class="el" href="dir_e7295b03dab2e9cdf32139bd8ec2e607.html">include</a></li><li class="navelem"><a class="el" href="dir_657344ecc65cfc28732701509f8d8421.html">NeuZephyr</a></li>  </ul>
</div>
</div><!-- top -->
<div id="doc-content">
<div class="header">
  <div class="summary">
<a href="#namespaces">Namespaces</a> &#124;
<a href="#func-members">Functions</a>  </div>
  <div class="headertitle"><div class="title">OperationKernels.cuh File Reference</div></div>
</div><!--header-->
<div class="contents">

<p>CUDA Kernel Definitions for High-Performance Tensor Operations.  
<a href="#details">More...</a></p>
<div class="textblock"><code>#include &lt;vector&gt;</code><br />
<code>#include &quot;Dimension.cuh&quot;</code><br />
</div><div class="textblock"><div class="dynheader">
Include dependency graph for OperationKernels.cuh:</div>
<div class="dyncontent">
<div class="center"><img src="_operation_kernels_8cuh__incl.png" border="0" usemap="#a_d_1_2_users_2_mgepahmge_2_documents_2_c_01_program_2_neu_zephyr_2include_2_neu_zephyr_2_operation_kernels_8cuh" alt=""/></div>
<map name="a_d_1_2_users_2_mgepahmge_2_documents_2_c_01_program_2_neu_zephyr_2include_2_neu_zephyr_2_operation_kernels_8cuh" id="a_d_1_2_users_2_mgepahmge_2_documents_2_c_01_program_2_neu_zephyr_2include_2_neu_zephyr_2_operation_kernels_8cuh">
<area shape="rect" title="CUDA Kernel Definitions for High&#45;Performance Tensor Operations." alt="" coords="5,5,208,80"/>
<area shape="rect" title=" " alt="" coords="32,203,90,229"/>
<area shape="poly" title=" " alt="" coords="99,81,71,188,66,187,94,80"/>
<area shape="rect" href="_dimension_8cuh_source.html" title=" " alt="" coords="99,128,207,155"/>
<area shape="poly" title=" " alt="" coords="127,79,143,113,138,115,122,81"/>
<area shape="poly" title=" " alt="" coords="139,157,90,195,87,191,135,153"/>
<area shape="rect" title=" " alt="" coords="114,203,185,229"/>
<area shape="poly" title=" " alt="" coords="155,155,153,187,148,187,150,155"/>
<area shape="rect" title=" " alt="" coords="209,203,287,229"/>
<area shape="poly" title=" " alt="" coords="171,153,221,191,218,195,168,157"/>
<area shape="rect" href="dl__export_8cuh_source.html" title=" " alt="" coords="312,203,411,229"/>
<area shape="poly" title=" " alt="" coords="190,153,311,195,310,200,189,158"/>
</map>
</div>
</div><div class="textblock"><div class="dynheader">
This graph shows which files directly or indirectly include this file:</div>
<div class="dyncontent">
<div class="center"><img src="_operation_kernels_8cuh__dep__incl.png" border="0" usemap="#a_d_1_2_users_2_mgepahmge_2_documents_2_c_01_program_2_neu_zephyr_2include_2_neu_zephyr_2_operation_kernels_8cuhdep" alt=""/></div>
<map name="a_d_1_2_users_2_mgepahmge_2_documents_2_c_01_program_2_neu_zephyr_2include_2_neu_zephyr_2_operation_kernels_8cuhdep" id="a_d_1_2_users_2_mgepahmge_2_documents_2_c_01_program_2_neu_zephyr_2include_2_neu_zephyr_2_operation_kernels_8cuhdep">
<area shape="rect" title="CUDA Kernel Definitions for High&#45;Performance Tensor Operations." alt="" coords="831,5,1033,80"/>
<area shape="rect" href="_compute_graph_8cuh.html" title="Defines the ComputeGraph class for constructing and managing computational graphs in neural network m..." alt="" coords="241,128,444,203"/>
<area shape="poly" title=" " alt="" coords="816,70,444,146,443,141,815,65"/>
<area shape="rect" href="_tensor_operations_8cuh_source.html" title=" " alt="" coords="761,128,964,203"/>
<area shape="poly" title=" " alt="" coords="905,95,886,129,881,126,901,92"/>
<area shape="rect" href="_mapped_tensor_8cu_source.html" title=" " alt="" coords="477,259,696,317"/>
<area shape="poly" title=" " alt="" coords="817,86,775,106,736,130,698,161,663,196,609,260,605,256,659,192,694,157,733,126,773,102,815,81"/>
<area shape="rect" href="_nodes_8cu_source.html" title=" " alt="" coords="720,259,891,317"/>
<area shape="poly" title=" " alt="" coords="818,76,780,98,751,129,744,145,742,161,748,196,764,229,783,257,779,260,759,231,743,197,737,161,739,143,747,127,777,94,815,71"/>
<area shape="rect" href="_tensor_8cu_source.html" title=" " alt="" coords="915,259,1087,317"/>
<area shape="poly" title=" " alt="" coords="964,93,978,127,995,197,1002,258,997,259,990,198,973,129,959,95"/>
<area shape="rect" href="_tensor_operations_8cu_source.html" title=" " alt="" coords="1112,259,1347,317"/>
<area shape="poly" title=" " alt="" coords="975,90,1032,154,1080,201,1124,231,1169,256,1166,260,1121,235,1077,205,1029,157,971,94"/>
<area shape="rect" href="_operation_kernels_8cu_source.html" title=" " alt="" coords="1090,136,1323,195"/>
<area shape="poly" title=" " alt="" coords="1031,84,1142,133,1140,138,1029,89"/>
<area shape="rect" href="_optimizer_8cu_source.html" title=" " alt="" coords="1348,136,1537,195"/>
<area shape="poly" title=" " alt="" coords="1049,62,1187,89,1337,125,1362,133,1361,138,1335,131,1186,95,1048,67"/>
<area shape="rect" href="_model_8cuh.html" title="Core class for computational graph construction and neural network modeling." alt="" coords="5,251,208,325"/>
<area shape="poly" title=" " alt="" coords="258,213,180,252,177,248,256,208"/>
<area shape="rect" href="_compute_graph_8cu_source.html" title=" " alt="" coords="232,259,453,317"/>
<area shape="poly" title=" " alt="" coords="345,218,345,258,340,258,340,218"/>
<area shape="rect" href="_model_8cu_source.html" title=" " alt="" coords="22,373,191,432"/>
<area shape="poly" title=" " alt="" coords="109,341,109,373,104,373,104,341"/>
<area shape="poly" title=" " alt="" coords="765,212,653,260,651,256,763,207"/>
<area shape="poly" title=" " alt="" coords="841,218,821,259,816,257,836,216"/>
<area shape="poly" title=" " alt="" coords="918,211,970,256,967,260,914,215"/>
<area shape="poly" title=" " alt="" coords="979,202,1142,255,1141,261,978,207"/>
</map>
</div>
</div>
<p><a href="_operation_kernels_8cuh_source.html">Go to the source code of this file.</a></p>
<table class="memberdecls">
<tr class="heading"><td colspan="2"><h2 class="groupheader"><a id="namespaces" name="namespaces"></a>
Namespaces</h2></td></tr>
<tr class="memitem:"><td class="memItemLeft" align="right" valign="top">namespace &#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html">nz::krnl</a></td></tr>
<tr class="memdesc:namespacenz_1_1krnl"><td class="mdescLeft">&#160;</td><td class="mdescRight">High-Performance CUDA Kernel Implementations for Tensor Computations. <br /></td></tr>
<tr class="separator:"><td class="memSeparator" colspan="2">&#160;</td></tr>
</table><table class="memberdecls">
<tr class="heading"><td colspan="2"><h2 class="groupheader"><a id="func-members" name="func-members"></a>
Functions</h2></td></tr>
<tr class="memitem:a97cda6dfc6545efaee2b686eed9ae766" id="r_a97cda6dfc6545efaee2b686eed9ae766"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#a97cda6dfc6545efaee2b686eed9ae766">nz::krnl::MatrixAdd</a> (dim3 gridDim, dim3 blockDim, float *a, float *b, float *c, unsigned long long n, size_t offset_c=0, size_t offset_a=0, size_t offset_b=0)</td></tr>
<tr class="memdesc:a97cda6dfc6545efaee2b686eed9ae766"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to perform matrix addition on GPU.  <br /></td></tr>
<tr class="separator:a97cda6dfc6545efaee2b686eed9ae766"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a5b29c405a1df9534430ad8682960ebb5" id="r_a5b29c405a1df9534430ad8682960ebb5"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#a5b29c405a1df9534430ad8682960ebb5">nz::krnl::MatrixAdd</a> (dim3 gridDim, dim3 blockDim, float *a, float *b, float *c, unsigned long long n, const std::vector&lt; size_t &gt; &amp;offset_c, const std::vector&lt; size_t &gt; &amp;offset_a, const std::vector&lt; size_t &gt; &amp;offset_b)</td></tr>
<tr class="memdesc:a5b29c405a1df9534430ad8682960ebb5"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to perform matrix addition on GPU.  <br /></td></tr>
<tr class="separator:a5b29c405a1df9534430ad8682960ebb5"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:ad18a2b0efc0cdfc9cb861396ad4da53f" id="r_ad18a2b0efc0cdfc9cb861396ad4da53f"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#ad18a2b0efc0cdfc9cb861396ad4da53f">nz::krnl::MatrixSub</a> (dim3 gridDim, dim3 blockDim, float *a, float *b, float *c, unsigned long long n, size_t offset_c=0, size_t offset_a=0, size_t offset_b=0)</td></tr>
<tr class="memdesc:ad18a2b0efc0cdfc9cb861396ad4da53f"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to perform matrix subtraction on GPU.  <br /></td></tr>
<tr class="separator:ad18a2b0efc0cdfc9cb861396ad4da53f"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a4ca041c74dc55e3ac9124b5fd39b985c" id="r_a4ca041c74dc55e3ac9124b5fd39b985c"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#a4ca041c74dc55e3ac9124b5fd39b985c">nz::krnl::MatrixSub</a> (dim3 gridDim, dim3 blockDim, float *a, float *b, float *c, unsigned long long n, const std::vector&lt; size_t &gt; &amp;offset_c, const std::vector&lt; size_t &gt; &amp;offset_a, const std::vector&lt; size_t &gt; &amp;offset_b)</td></tr>
<tr class="memdesc:a4ca041c74dc55e3ac9124b5fd39b985c"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to perform matrix subtraction on GPU.  <br /></td></tr>
<tr class="separator:a4ca041c74dc55e3ac9124b5fd39b985c"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:ae30a6e1de69588aa0c6eb8a5b8e6e826" id="r_ae30a6e1de69588aa0c6eb8a5b8e6e826"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#ae30a6e1de69588aa0c6eb8a5b8e6e826">nz::krnl::GeneralMatrixMul</a> (dim3 gridDim, dim3 blockDim, float *A, float *B, float *C, unsigned long long M, unsigned long long N, unsigned long long K, size_t offset_c=0, size_t offset_a=0, size_t offset_b=0)</td></tr>
<tr class="memdesc:ae30a6e1de69588aa0c6eb8a5b8e6e826"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to perform single-precision matrix multiplication on GPU using CUDA cores.  <br /></td></tr>
<tr class="separator:ae30a6e1de69588aa0c6eb8a5b8e6e826"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:aa3720ebf4ae0cc9f4abbd1e32842191b" id="r_aa3720ebf4ae0cc9f4abbd1e32842191b"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#aa3720ebf4ae0cc9f4abbd1e32842191b">nz::krnl::GeneralMatrixMul</a> (dim3 gridDim, dim3 blockDim, float *A, float *B, float *C, unsigned long long M, unsigned long long N, unsigned long long K, const std::vector&lt; size_t &gt; &amp;offset_c, const std::vector&lt; size_t &gt; &amp;offset_a, const std::vector&lt; size_t &gt; &amp;offset_b)</td></tr>
<tr class="memdesc:aa3720ebf4ae0cc9f4abbd1e32842191b"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to perform single-precision matrix multiplication on GPU using CUDA cores.  <br /></td></tr>
<tr class="separator:aa3720ebf4ae0cc9f4abbd1e32842191b"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:afe3f38f788c735b7eb718443eb0fd094" id="r_afe3f38f788c735b7eb718443eb0fd094"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#afe3f38f788c735b7eb718443eb0fd094">nz::krnl::Transpose</a> (dim3 gridDim, dim3 blockDim, float *d_A, float *d_B, unsigned int rows, unsigned int cols, size_t offset=0)</td></tr>
<tr class="memdesc:afe3f38f788c735b7eb718443eb0fd094"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to transpose a matrix on the GPU.  <br /></td></tr>
<tr class="separator:afe3f38f788c735b7eb718443eb0fd094"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a16823e30ad99965b64a03e2d4a91a699" id="r_a16823e30ad99965b64a03e2d4a91a699"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#a16823e30ad99965b64a03e2d4a91a699">nz::krnl::Transpose</a> (dim3 gridDim, dim3 blockDim, float *d_A, float *d_B, unsigned int rows, unsigned int cols, const std::vector&lt; size_t &gt; &amp;offset)</td></tr>
<tr class="memdesc:a16823e30ad99965b64a03e2d4a91a699"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to transpose a matrix on the GPU.  <br /></td></tr>
<tr class="separator:a16823e30ad99965b64a03e2d4a91a699"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a5af716524e248c61f3dce227d8ef6e34" id="r_a5af716524e248c61f3dce227d8ef6e34"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#a5af716524e248c61f3dce227d8ef6e34">nz::krnl::ScalarMul</a> (dim3 gridDim, dim3 blockDim, float *out, float *in, float num, unsigned long long n)</td></tr>
<tr class="memdesc:a5af716524e248c61f3dce227d8ef6e34"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to perform scalar multiplication on the GPU.  <br /></td></tr>
<tr class="separator:a5af716524e248c61f3dce227d8ef6e34"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a27bc4025be4253d5fffae2bf1b43b3af" id="r_a27bc4025be4253d5fffae2bf1b43b3af"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#a27bc4025be4253d5fffae2bf1b43b3af">nz::krnl::ScalarDiv</a> (dim3 gridDim, dim3 blockDim, float *out, float *in, float num, unsigned long long n)</td></tr>
<tr class="memdesc:a27bc4025be4253d5fffae2bf1b43b3af"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to perform scalar division on the GPU.  <br /></td></tr>
<tr class="separator:a27bc4025be4253d5fffae2bf1b43b3af"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a56f84e531825be8b2b0974c2488eb765" id="r_a56f84e531825be8b2b0974c2488eb765"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#a56f84e531825be8b2b0974c2488eb765">nz::krnl::ScalarAdd</a> (dim3 gridDim, dim3 blockDim, float *out, float *in, float num, unsigned long long n)</td></tr>
<tr class="memdesc:a56f84e531825be8b2b0974c2488eb765"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to add a scalar to each element of a matrix on the GPU.  <br /></td></tr>
<tr class="separator:a56f84e531825be8b2b0974c2488eb765"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:af7069a420e81babb49b1bc009333d053" id="r_af7069a420e81babb49b1bc009333d053"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#af7069a420e81babb49b1bc009333d053">nz::krnl::Negation</a> (dim3 gridDim, dim3 blockDim, float *out, float *in, unsigned long long n)</td></tr>
<tr class="memdesc:af7069a420e81babb49b1bc009333d053"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to negate each element of a matrix on the GPU.  <br /></td></tr>
<tr class="separator:af7069a420e81babb49b1bc009333d053"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:adc047e65307dbc711235f637227b7d10" id="r_adc047e65307dbc711235f637227b7d10"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#adc047e65307dbc711235f637227b7d10">nz::krnl::Recip</a> (dim3 gridDim, dim3 blockDim, float *out, float *in, unsigned long long n)</td></tr>
<tr class="memdesc:adc047e65307dbc711235f637227b7d10"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to compute the reciprocal of each element of a matrix on the GPU.  <br /></td></tr>
<tr class="separator:adc047e65307dbc711235f637227b7d10"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a8855f411733f7de29d013f4ad40096c9" id="r_a8855f411733f7de29d013f4ad40096c9"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#a8855f411733f7de29d013f4ad40096c9">nz::krnl::RectifiedLinearUnit</a> (dim3 gridDim, dim3 blockDim, float *out, float *in, unsigned long long n)</td></tr>
<tr class="memdesc:a8855f411733f7de29d013f4ad40096c9"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to apply the Rectified Linear Unit (ReLU) activation on the GPU.  <br /></td></tr>
<tr class="separator:a8855f411733f7de29d013f4ad40096c9"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a4ddfc808de99fe831e74a3bd3f9bbdaf" id="r_a4ddfc808de99fe831e74a3bd3f9bbdaf"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#a4ddfc808de99fe831e74a3bd3f9bbdaf">nz::krnl::ReLUBackward</a> (dim3 gridDim, dim3 blockDim, float *A_grad, float *A, float *B_grad, unsigned long long n)</td></tr>
<tr class="memdesc:a4ddfc808de99fe831e74a3bd3f9bbdaf"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to compute the gradient of the ReLU activation during backpropagation.  <br /></td></tr>
<tr class="separator:a4ddfc808de99fe831e74a3bd3f9bbdaf"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a21bbbcf6d97bfaccc828ce7736814bd4" id="r_a21bbbcf6d97bfaccc828ce7736814bd4"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#a21bbbcf6d97bfaccc828ce7736814bd4">nz::krnl::Sigmoid</a> (dim3 gridDim, dim3 blockDim, float *out, float *in, unsigned long long n)</td></tr>
<tr class="memdesc:a21bbbcf6d97bfaccc828ce7736814bd4"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to apply the Sigmoid activation function on the GPU.  <br /></td></tr>
<tr class="separator:a21bbbcf6d97bfaccc828ce7736814bd4"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:aff1f9f1bf9fb677024bd2b565fab9801" id="r_aff1f9f1bf9fb677024bd2b565fab9801"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#aff1f9f1bf9fb677024bd2b565fab9801">nz::krnl::SigmoidBackward</a> (dim3 gridDim, dim3 blockDim, float *A_grad, float *B, float *B_grad, unsigned long long n)</td></tr>
<tr class="memdesc:aff1f9f1bf9fb677024bd2b565fab9801"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to compute the gradient of the Sigmoid activation during backpropagation.  <br /></td></tr>
<tr class="separator:aff1f9f1bf9fb677024bd2b565fab9801"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:aeb7d10939b25508e0b5db1fe44f4b467" id="r_aeb7d10939b25508e0b5db1fe44f4b467"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#aeb7d10939b25508e0b5db1fe44f4b467">nz::krnl::Tanh</a> (dim3 gridDim, dim3 blockDim, float *out, float *in, unsigned long long n)</td></tr>
<tr class="memdesc:aeb7d10939b25508e0b5db1fe44f4b467"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to apply the Tanh activation function on the GPU.  <br /></td></tr>
<tr class="separator:aeb7d10939b25508e0b5db1fe44f4b467"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a90d501e72361b7341f36394af0f27c74" id="r_a90d501e72361b7341f36394af0f27c74"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#a90d501e72361b7341f36394af0f27c74">nz::krnl::TanhBackward</a> (dim3 gridDim, dim3 blockDim, float *A_grad, float *B, float *B_grad, unsigned long long n)</td></tr>
<tr class="memdesc:a90d501e72361b7341f36394af0f27c74"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to compute the gradient of the Tanh activation during backpropagation.  <br /></td></tr>
<tr class="separator:a90d501e72361b7341f36394af0f27c74"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a04246c5218530f789a0ed4811b7ef3f3" id="r_a04246c5218530f789a0ed4811b7ef3f3"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#a04246c5218530f789a0ed4811b7ef3f3">nz::krnl::LeakyReLU</a> (dim3 gridDim, dim3 blockDim, float *out, float *in, unsigned long long n, float alpha=0.01f)</td></tr>
<tr class="memdesc:a04246c5218530f789a0ed4811b7ef3f3"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to apply the Leaky ReLU activation function on the GPU.  <br /></td></tr>
<tr class="separator:a04246c5218530f789a0ed4811b7ef3f3"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a7eade95ddcf48141d69bb19803b22d51" id="r_a7eade95ddcf48141d69bb19803b22d51"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#a7eade95ddcf48141d69bb19803b22d51">nz::krnl::LeakyReLUBackward</a> (dim3 gridDim, dim3 blockDim, float *A_grad, float *A, float *B_grad, unsigned long long n, float alpha=0.01f)</td></tr>
<tr class="memdesc:a7eade95ddcf48141d69bb19803b22d51"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to compute the gradient of the Leaky ReLU activation during backpropagation.  <br /></td></tr>
<tr class="separator:a7eade95ddcf48141d69bb19803b22d51"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a997aa5460fd64fadf9b701fbf73e3fb2" id="r_a997aa5460fd64fadf9b701fbf73e3fb2"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#a997aa5460fd64fadf9b701fbf73e3fb2">nz::krnl::Swish</a> (dim3 gridDim, dim3 blockDim, float *out, float *in, unsigned long long n)</td></tr>
<tr class="memdesc:a997aa5460fd64fadf9b701fbf73e3fb2"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to apply the Swish activation function on the GPU.  <br /></td></tr>
<tr class="separator:a997aa5460fd64fadf9b701fbf73e3fb2"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a6c5a4b54442aab42df5afe8688e71596" id="r_a6c5a4b54442aab42df5afe8688e71596"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#a6c5a4b54442aab42df5afe8688e71596">nz::krnl::SwishBackward</a> (dim3 gridDim, dim3 blockDim, float *A_grad, float *A, float *B, float *B_grad, unsigned long long n)</td></tr>
<tr class="memdesc:a6c5a4b54442aab42df5afe8688e71596"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to compute the gradient of the Swish activation during backpropagation.  <br /></td></tr>
<tr class="separator:a6c5a4b54442aab42df5afe8688e71596"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a0e82aca250b46ac8ded8cae8936d7e38" id="r_a0e82aca250b46ac8ded8cae8936d7e38"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#a0e82aca250b46ac8ded8cae8936d7e38">nz::krnl::ExponentialLinearUnit</a> (dim3 gridDim, dim3 blockDim, float *out, float *in, unsigned long long n, float alpha=1.0f)</td></tr>
<tr class="memdesc:a0e82aca250b46ac8ded8cae8936d7e38"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to apply the Exponential Linear Unit (ELU) activation function on the GPU.  <br /></td></tr>
<tr class="separator:a0e82aca250b46ac8ded8cae8936d7e38"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:aee8ca471aa260bd1fca5b1797e229f9f" id="r_aee8ca471aa260bd1fca5b1797e229f9f"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#aee8ca471aa260bd1fca5b1797e229f9f">nz::krnl::ELUBackward</a> (dim3 gridDim, dim3 blockDim, float *A_grad, float *A, float *B_grad, unsigned long long n, float alpha=1.0f)</td></tr>
<tr class="memdesc:aee8ca471aa260bd1fca5b1797e229f9f"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to compute the gradient of the ELU activation during backpropagation.  <br /></td></tr>
<tr class="separator:aee8ca471aa260bd1fca5b1797e229f9f"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a52e449285e560185378234aecaf2f87c" id="r_a52e449285e560185378234aecaf2f87c"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#a52e449285e560185378234aecaf2f87c">nz::krnl::HardSigmoid</a> (dim3 gridDim, dim3 blockDim, float *out, float *in, unsigned long long n, float alpha=0.2f, float beta=0.5f)</td></tr>
<tr class="memdesc:a52e449285e560185378234aecaf2f87c"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to apply the Hard Sigmoid activation function on the GPU.  <br /></td></tr>
<tr class="separator:a52e449285e560185378234aecaf2f87c"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a43232f9472ad3b974351e59386208efa" id="r_a43232f9472ad3b974351e59386208efa"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#a43232f9472ad3b974351e59386208efa">nz::krnl::HardSigmoidBackward</a> (dim3 gridDim, dim3 blockDim, float *A_grad, float *A, float *B_grad, unsigned long long n, float alpha=0.2f, float beta=0.5f)</td></tr>
<tr class="memdesc:a43232f9472ad3b974351e59386208efa"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to compute the gradient of the Hard Sigmoid activation during backpropagation.  <br /></td></tr>
<tr class="separator:a43232f9472ad3b974351e59386208efa"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:aef9c028ed356b5684e103639bb23bcf0" id="r_aef9c028ed356b5684e103639bb23bcf0"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#aef9c028ed356b5684e103639bb23bcf0">nz::krnl::HardSwish</a> (dim3 gridDim, dim3 blockDim, float *out, float *in, unsigned long long n, float alpha=0.2f, float beta=0.5f)</td></tr>
<tr class="memdesc:aef9c028ed356b5684e103639bb23bcf0"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to apply the Hard Swish activation function on the GPU.  <br /></td></tr>
<tr class="separator:aef9c028ed356b5684e103639bb23bcf0"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a455365870d43ff26687a731d15c4cdff" id="r_a455365870d43ff26687a731d15c4cdff"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#a455365870d43ff26687a731d15c4cdff">nz::krnl::HardSwishBackward</a> (dim3 gridDim, dim3 blockDim, float *A_grad, float *A, float *B_grad, unsigned long long n, float alpha=0.2f, float beta=0.5f)</td></tr>
<tr class="memdesc:a455365870d43ff26687a731d15c4cdff"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to compute the gradient of the Hard Swish activation during backpropagation.  <br /></td></tr>
<tr class="separator:a455365870d43ff26687a731d15c4cdff"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a51a5ff3c8cc2c3051fddf32de294b467" id="r_a51a5ff3c8cc2c3051fddf32de294b467"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#a51a5ff3c8cc2c3051fddf32de294b467">nz::krnl::SummationExp</a> (dim3 gridDim, dim3 blockDim, size_t sharedMemSize, float *out, float *g_data, unsigned long long n, size_t offset=0)</td></tr>
<tr class="memdesc:a51a5ff3c8cc2c3051fddf32de294b467"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to compute the summation of exponentials of each element in the input array.  <br /></td></tr>
<tr class="separator:a51a5ff3c8cc2c3051fddf32de294b467"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:adbafc409d57fa0a9d78ecac5bf7b10a3" id="r_adbafc409d57fa0a9d78ecac5bf7b10a3"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#adbafc409d57fa0a9d78ecac5bf7b10a3">nz::krnl::Softmax</a> (dim3 gridDim, dim3 blockDim, float *out, float *in, float exp_sum_of_input, unsigned long long n, size_t offset=0)</td></tr>
<tr class="memdesc:adbafc409d57fa0a9d78ecac5bf7b10a3"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to apply the Softmax function on the GPU.  <br /></td></tr>
<tr class="separator:adbafc409d57fa0a9d78ecac5bf7b10a3"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a4375738c83ef892783abc210578e5b39" id="r_a4375738c83ef892783abc210578e5b39"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#a4375738c83ef892783abc210578e5b39">nz::krnl::SoftmaxJacobian</a> (dim3 gridDim, dim3 blockDim, float *out, float *in, unsigned long long n)</td></tr>
<tr class="memdesc:a4375738c83ef892783abc210578e5b39"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to compute the Jacobian of the Softmax function.  <br /></td></tr>
<tr class="separator:a4375738c83ef892783abc210578e5b39"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:af76ce6a930db4def5ceb51350af72f3c" id="r_af76ce6a930db4def5ceb51350af72f3c"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#af76ce6a930db4def5ceb51350af72f3c">nz::krnl::MeanSquaredError</a> (dim3 gridDim, dim3 blockDim, size_t sharedMemSize, float *out, float *predict, float *real, unsigned long long n)</td></tr>
<tr class="memdesc:af76ce6a930db4def5ceb51350af72f3c"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to compute the Mean Squared Error (MSE) loss between predicted and real values.  <br /></td></tr>
<tr class="separator:af76ce6a930db4def5ceb51350af72f3c"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:ae77920db6adf79a17dbfb1dbf1ab5656" id="r_ae77920db6adf79a17dbfb1dbf1ab5656"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#ae77920db6adf79a17dbfb1dbf1ab5656">nz::krnl::MSEBackward</a> (dim3 gridDim, dim3 blockDim, float *out, float *predict, float *real, unsigned long long n)</td></tr>
<tr class="memdesc:ae77920db6adf79a17dbfb1dbf1ab5656"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to compute the gradient of the Mean Squared Error (MSE) loss for backpropagation.  <br /></td></tr>
<tr class="separator:ae77920db6adf79a17dbfb1dbf1ab5656"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:aeec286d5351eee7061e151470adb4eef" id="r_aeec286d5351eee7061e151470adb4eef"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#aeec286d5351eee7061e151470adb4eef">nz::krnl::StochasticGradientDescent</a> (dim3 gridDim, dim3 blockDim, float *data, float *grad, float lr, unsigned long long n)</td></tr>
<tr class="memdesc:aeec286d5351eee7061e151470adb4eef"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to perform Stochastic Gradient Descent (SGD) optimization.  <br /></td></tr>
<tr class="separator:aeec286d5351eee7061e151470adb4eef"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:abf927faf0950fbc215564c67b8ac57be" id="r_abf927faf0950fbc215564c67b8ac57be"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#abf927faf0950fbc215564c67b8ac57be">nz::krnl::BinaryCrossEntropy</a> (dim3 gridDim, dim3 blockDim, size_t sharedMemSize, float *out, float *predict, float *real, unsigned long long n)</td></tr>
<tr class="memdesc:abf927faf0950fbc215564c67b8ac57be"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to compute the Binary Cross Entropy (BCE) loss between predicted and real values.  <br /></td></tr>
<tr class="separator:abf927faf0950fbc215564c67b8ac57be"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a1fc3d553947a5cad87f29989f9d9465d" id="r_a1fc3d553947a5cad87f29989f9d9465d"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#a1fc3d553947a5cad87f29989f9d9465d">nz::krnl::BCEBackward</a> (dim3 gridDim, dim3 blockDim, float *out, float *predict, float *real, unsigned long long n)</td></tr>
<tr class="memdesc:a1fc3d553947a5cad87f29989f9d9465d"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to compute the gradient of Binary Cross Entropy (BCE) loss for backpropagation.  <br /></td></tr>
<tr class="separator:a1fc3d553947a5cad87f29989f9d9465d"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a273ef3023442a864f1028becaf236bae" id="r_a273ef3023442a864f1028becaf236bae"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#a273ef3023442a864f1028becaf236bae">nz::krnl::Momentum</a> (dim3 gridDim, dim3 blockDim, float *output, float *grad, float *velocity, float beta, unsigned long long n)</td></tr>
<tr class="memdesc:a273ef3023442a864f1028becaf236bae"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to apply Momentum optimization.  <br /></td></tr>
<tr class="separator:a273ef3023442a864f1028becaf236bae"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a1e915bd4a354938d8bc2d09be00eae76" id="r_a1e915bd4a354938d8bc2d09be00eae76"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#a1e915bd4a354938d8bc2d09be00eae76">nz::krnl::AdaGrad</a> (dim3 gridDim, dim3 blockDim, float *data, float *G, float *grad, float lr, float eps, unsigned long long n)</td></tr>
<tr class="memdesc:a1e915bd4a354938d8bc2d09be00eae76"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to apply AdaGrad optimization.  <br /></td></tr>
<tr class="separator:a1e915bd4a354938d8bc2d09be00eae76"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:aaf3c9cca114d003130ffa4354b4a24de" id="r_aaf3c9cca114d003130ffa4354b4a24de"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#aaf3c9cca114d003130ffa4354b4a24de">nz::krnl::RMSprop</a> (dim3 gridDim, dim3 blockDim, float *data, float *v, float *grad, float lr, float beta, float eps, unsigned long long n)</td></tr>
<tr class="memdesc:aaf3c9cca114d003130ffa4354b4a24de"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to apply RMSprop optimization.  <br /></td></tr>
<tr class="separator:aaf3c9cca114d003130ffa4354b4a24de"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a2b9ab840eeb0e74f4b78277a046b3a07" id="r_a2b9ab840eeb0e74f4b78277a046b3a07"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#a2b9ab840eeb0e74f4b78277a046b3a07">nz::krnl::Adam</a> (dim3 gridDim, dim3 blockDim, float *data, float *m, float *v, float *grad, float lr, float beta1, float beta2, float eps, int t, unsigned long long n)</td></tr>
<tr class="memdesc:a2b9ab840eeb0e74f4b78277a046b3a07"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to apply Adam optimization.  <br /></td></tr>
<tr class="separator:a2b9ab840eeb0e74f4b78277a046b3a07"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:ada94b8c5c6e6d72132face63a3305624" id="r_ada94b8c5c6e6d72132face63a3305624"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#ada94b8c5c6e6d72132face63a3305624">nz::krnl::NAdam</a> (dim3 gridDim, dim3 blockDim, float *data, float *m, float *m_modified, float *v, float *grad, float lr, float beta1, float beta2, float eps, int t, unsigned long long n)</td></tr>
<tr class="memdesc:ada94b8c5c6e6d72132face63a3305624"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to apply NAdam optimization.  <br /></td></tr>
<tr class="separator:ada94b8c5c6e6d72132face63a3305624"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a1f71726879c2d6a9d790522cdc1576e1" id="r_a1f71726879c2d6a9d790522cdc1576e1"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#a1f71726879c2d6a9d790522cdc1576e1">nz::krnl::AdaDelta</a> (dim3 gridDim, dim3 blockDim, float *data, float *acc_delta, float *acc_grad, float *grad, float rho, float eps, unsigned long long n)</td></tr>
<tr class="memdesc:a1f71726879c2d6a9d790522cdc1576e1"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to apply AdaDelta optimization.  <br /></td></tr>
<tr class="separator:a1f71726879c2d6a9d790522cdc1576e1"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:aa84aa2397f4f5a09a96bef76726e46f0" id="r_aa84aa2397f4f5a09a96bef76726e46f0"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#aa84aa2397f4f5a09a96bef76726e46f0">nz::krnl::TensorCoreGEMM</a> (float *A, float *B, float *C, unsigned long long M, unsigned long long N, unsigned long long K)</td></tr>
<tr class="memdesc:aa84aa2397f4f5a09a96bef76726e46f0"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to perform fast matrix multiplication using Tensor Cores with half-precision (FP16) support.  <br /></td></tr>
<tr class="separator:aa84aa2397f4f5a09a96bef76726e46f0"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:ad136c8a6560a5305984ce0a31bea71bf" id="r_ad136c8a6560a5305984ce0a31bea71bf"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#ad136c8a6560a5305984ce0a31bea71bf">nz::krnl::Fill</a> (dim3 gridDim, dim3 blockDim, float *data, float value, unsigned long long n, size_t offset=0)</td></tr>
<tr class="memdesc:ad136c8a6560a5305984ce0a31bea71bf"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to fill a data array with a given value.  <br /></td></tr>
<tr class="separator:ad136c8a6560a5305984ce0a31bea71bf"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a8ec4524fdefd3d771c72e77e94281c88" id="r_a8ec4524fdefd3d771c72e77e94281c88"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#a8ec4524fdefd3d771c72e77e94281c88">nz::krnl::HadamardProduct</a> (dim3 gridDim, dim3 blockDim, float *out, float *in1, float *in2, unsigned long long n)</td></tr>
<tr class="memdesc:a8ec4524fdefd3d771c72e77e94281c88"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to perform element-wise Hadamard product of two arrays.  <br /></td></tr>
<tr class="separator:a8ec4524fdefd3d771c72e77e94281c88"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:aa61cded4977bb2dc3720f7057cc2fb47" id="r_aa61cded4977bb2dc3720f7057cc2fb47"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#aa61cded4977bb2dc3720f7057cc2fb47">nz::krnl::ElementwiseDivide</a> (dim3 gridDim, dim3 blockDim, float *out, float *in1, float *in2, unsigned long long n, size_t offset_o=0, size_t offset_1=0, size_t offset_2=0)</td></tr>
<tr class="memdesc:aa61cded4977bb2dc3720f7057cc2fb47"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to perform element-wise division of two arrays.  <br /></td></tr>
<tr class="separator:aa61cded4977bb2dc3720f7057cc2fb47"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a1ae846a65c2f5b83cd1b9fc61b877854" id="r_a1ae846a65c2f5b83cd1b9fc61b877854"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#a1ae846a65c2f5b83cd1b9fc61b877854">nz::krnl::Summation</a> (dim3 gridDim, dim3 blockDim, unsigned long long sharedMemSize, float *out, float *in, unsigned long long n, size_t offset=0)</td></tr>
<tr class="memdesc:a1ae846a65c2f5b83cd1b9fc61b877854"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to perform element-wise summation of two arrays.  <br /></td></tr>
<tr class="separator:a1ae846a65c2f5b83cd1b9fc61b877854"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a0ed44a68bfb86a9fd3d6c3b25614713f" id="r_a0ed44a68bfb86a9fd3d6c3b25614713f"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#a0ed44a68bfb86a9fd3d6c3b25614713f">nz::krnl::gradCopy</a> (dim3 gridDim, dim3 blockDim, float *out, float *in, size_t n, const std::vector&lt; size_t &gt; &amp;offset_o, const std::vector&lt; size_t &gt; &amp;offset_i)</td></tr>
<tr class="memdesc:a0ed44a68bfb86a9fd3d6c3b25614713f"><td class="mdescLeft">&#160;</td><td class="mdescRight">Copies gradient data from one array to another with specified offsets.  <br /></td></tr>
<tr class="separator:a0ed44a68bfb86a9fd3d6c3b25614713f"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a9ac0590fbb5eb7f51b05da574e9845a8" id="r_a9ac0590fbb5eb7f51b05da574e9845a8"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#a9ac0590fbb5eb7f51b05da574e9845a8">nz::krnl::NgradCopy</a> (dim3 gridDim, dim3 blockDim, float *out, float *in, size_t n, const std::vector&lt; size_t &gt; &amp;offset_o, const std::vector&lt; size_t &gt; &amp;offset_i)</td></tr>
<tr class="memdesc:a9ac0590fbb5eb7f51b05da574e9845a8"><td class="mdescLeft">&#160;</td><td class="mdescRight">Copies gradient data from one array to another with specified offsets.  <br /></td></tr>
<tr class="separator:a9ac0590fbb5eb7f51b05da574e9845a8"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:ae45dbebceb76ddf82fa5e6b9df882e62" id="r_ae45dbebceb76ddf82fa5e6b9df882e62"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#ae45dbebceb76ddf82fa5e6b9df882e62">nz::krnl::Expand</a> (dim3 gridDim, dim3 blockDim, float *out, float *in, size_t n, size_t total)</td></tr>
<tr class="memdesc:ae45dbebceb76ddf82fa5e6b9df882e62"><td class="mdescLeft">&#160;</td><td class="mdescRight">Expands the input array into the output array with a specified total size.  <br /></td></tr>
<tr class="separator:ae45dbebceb76ddf82fa5e6b9df882e62"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a454a28ef0e22014efca1ede4e954db65" id="r_a454a28ef0e22014efca1ede4e954db65"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#a454a28ef0e22014efca1ede4e954db65">nz::krnl::Compress</a> (dim3 gridDim, dim3 blockDim, float *out, float *in, size_t n, size_t total)</td></tr>
<tr class="memdesc:a454a28ef0e22014efca1ede4e954db65"><td class="mdescLeft">&#160;</td><td class="mdescRight">Compresses the input array into the output array with a specified total size.  <br /></td></tr>
<tr class="separator:a454a28ef0e22014efca1ede4e954db65"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a3a781324400c54c35dd564f3599dca8e" id="r_a3a781324400c54c35dd564f3599dca8e"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#a3a781324400c54c35dd564f3599dca8e">nz::krnl::img2col</a> (dim3 gridDim, dim3 blockDim, float *out, float *in, size_t H_out, size_t W_out, size_t C, size_t K_h, size_t K_w, size_t stride, size_t pad, size_t H_in, size_t W_in, size_t batch)</td></tr>
<tr class="memdesc:a3a781324400c54c35dd564f3599dca8e"><td class="mdescLeft">&#160;</td><td class="mdescRight">Rearranges image data into column format for convolution operations.  <br /></td></tr>
<tr class="separator:a3a781324400c54c35dd564f3599dca8e"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a1c2b7a6f28d2af22f9a2623c5ae62bff" id="r_a1c2b7a6f28d2af22f9a2623c5ae62bff"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#a1c2b7a6f28d2af22f9a2623c5ae62bff">nz::krnl::img2colBackward</a> (dim3 gridDim, dim3 blockDim, float *out, float *in, size_t H_out, size_t W_out, size_t C, size_t K_h, size_t K_w, size_t stride, size_t pad, size_t H_in, size_t W_in, size_t batch)</td></tr>
<tr class="memdesc:a1c2b7a6f28d2af22f9a2623c5ae62bff"><td class="mdescLeft">&#160;</td><td class="mdescRight">Rearranges columnar data back into image format for backpropagation in convolution operations.  <br /></td></tr>
<tr class="separator:a1c2b7a6f28d2af22f9a2623c5ae62bff"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a7c061f5511c3ab9d36563757bd969ff7" id="r_a7c061f5511c3ab9d36563757bd969ff7"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#a7c061f5511c3ab9d36563757bd969ff7">nz::krnl::col2img</a> (dim3 gridDim, dim3 blockDim, float *out, float *in, size_t H_out, size_t W_out, size_t C_out, size_t batches)</td></tr>
<tr class="memdesc:a7c061f5511c3ab9d36563757bd969ff7"><td class="mdescLeft">&#160;</td><td class="mdescRight">Rearranges columnar data back into image format.  <br /></td></tr>
<tr class="separator:a7c061f5511c3ab9d36563757bd969ff7"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a028970809074d79f28ff94f62b3edaa4" id="r_a028970809074d79f28ff94f62b3edaa4"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#a028970809074d79f28ff94f62b3edaa4">nz::krnl::col2imgBackward</a> (dim3 gridDim, dim3 blockDim, float *out, float *in, size_t H_out, size_t W_out, size_t C_out, size_t batches)</td></tr>
<tr class="memdesc:a028970809074d79f28ff94f62b3edaa4"><td class="mdescLeft">&#160;</td><td class="mdescRight">Rearranges columnar data back into image format for backpropagation.  <br /></td></tr>
<tr class="separator:a028970809074d79f28ff94f62b3edaa4"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:addaa377a94d007df2690043b08904e28" id="r_addaa377a94d007df2690043b08904e28"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#addaa377a94d007df2690043b08904e28">nz::krnl::AveragePooling</a> (dim3 gridDim, dim3 blockDim, float *out, float *in, size_t pool_size, size_t stride, size_t padding, size_t batches, size_t channels, size_t H_in, size_t W_in, size_t H_out, size_t W_out)</td></tr>
<tr class="memdesc:addaa377a94d007df2690043b08904e28"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to perform average pooling on the GPU.  <br /></td></tr>
<tr class="separator:addaa377a94d007df2690043b08904e28"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a551402f9c55653c9fae63e172a5fb250" id="r_a551402f9c55653c9fae63e172a5fb250"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#a551402f9c55653c9fae63e172a5fb250">nz::krnl::AveragePoolingBackward</a> (dim3 gridDim, dim3 blockDim, float *out, float *in, size_t pool_size, size_t stride, size_t padding, size_t batches, size_t channels, size_t H_in, size_t W_in, size_t H_out, size_t W_out)</td></tr>
<tr class="memdesc:a551402f9c55653c9fae63e172a5fb250"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to compute the gradient of average pooling during backpropagation.  <br /></td></tr>
<tr class="separator:a551402f9c55653c9fae63e172a5fb250"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a73ceb77688c4008dc350fc87b99875aa" id="r_a73ceb77688c4008dc350fc87b99875aa"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#a73ceb77688c4008dc350fc87b99875aa">nz::krnl::GlobalAvgPoolBackward</a> (dim3 gridDim, dim3 blockDim, float *output, float *in, size_t batches, size_t channels, size_t height, size_t width)</td></tr>
<tr class="memdesc:a73ceb77688c4008dc350fc87b99875aa"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to compute the gradient of global average pooling during backpropagation.  <br /></td></tr>
<tr class="separator:a73ceb77688c4008dc350fc87b99875aa"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:abcc632e5a7104c1a28208e94a4ce6e28" id="r_abcc632e5a7104c1a28208e94a4ce6e28"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#abcc632e5a7104c1a28208e94a4ce6e28">nz::krnl::MaxPooling</a> (dim3 gridDim, dim3 blockDim, float *output, float *position, float *input, size_t pool_size, size_t stride, size_t padding, size_t batches, size_t channels, size_t H_in, size_t W_in, size_t H_out, size_t W_out)</td></tr>
<tr class="memdesc:abcc632e5a7104c1a28208e94a4ce6e28"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to perform max pooling on the GPU.  <br /></td></tr>
<tr class="separator:abcc632e5a7104c1a28208e94a4ce6e28"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a0d5f5f4c9e89a8d914a7f2f802d1caab" id="r_a0d5f5f4c9e89a8d914a7f2f802d1caab"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="namespacenz_1_1krnl.html#a0d5f5f4c9e89a8d914a7f2f802d1caab">nz::krnl::MaxPoolingBackward</a> (dim3 gridDim, dim3 blockDim, float *output, float *position, float *input, size_t pool_size, size_t stride, size_t padding, size_t batches, size_t channels, size_t H_in, size_t W_in, size_t H_out, size_t W_out)</td></tr>
<tr class="memdesc:a0d5f5f4c9e89a8d914a7f2f802d1caab"><td class="mdescLeft">&#160;</td><td class="mdescRight">Kernel function to compute the gradient of max pooling during backpropagation.  <br /></td></tr>
<tr class="separator:a0d5f5f4c9e89a8d914a7f2f802d1caab"><td class="memSeparator" colspan="2">&#160;</td></tr>
</table>
<a name="details" id="details"></a><h2 class="groupheader">Detailed Description</h2>
<div class="textblock"><p>CUDA Kernel Definitions for High-Performance Tensor Operations. </p>
<p>This header file provides a comprehensive collection of CUDA kernel functions for accelerated tensor computations, designed to support various mathematical operations, neural network layers, activation functions, and optimization algorithms.</p>
<p>The kernel functions in this file are organized within the <code><a class="el" href="namespacenz_1_1krnl.html" title="High-Performance CUDA Kernel Implementations for Tensor Computations.">nz::krnl</a></code> namespace and cover a wide range of computational tasks:</p>
<ul>
<li><b>Matrix Operations</b>: Basic matrix arithmetic like addition, subtraction, multiplication, and transposition.</li>
<li><b>Element-wise Operations</b>: Scalar operations, negation, reciprocal calculations.</li>
<li><b>Activation Functions</b>:<ul>
<li>Linear: ReLU, Leaky ReLU</li>
<li>Sigmoid Variants: Standard Sigmoid, Hard Sigmoid</li>
<li>Non-linear: Tanh, Swish, ELU</li>
</ul>
</li>
<li><b>Backward Propagation Kernels</b>: Gradient computations for each activation function.</li>
<li><b>Loss Functions</b>: Mean Squared Error, Binary Cross-Entropy</li>
<li><b>Optimization Algorithms</b>: Stochastic Gradient Descent, Momentum, AdaGrad, RMSprop, Adam, NAdam, AdaDelta</li>
</ul>
<p>Kernels are designed for parallel execution on CUDA-enabled GPUs, leveraging high-performance computing capabilities for efficient deep learning computations.</p>
<dl class="section note"><dt>Note</dt><dd><ul>
<li>All kernels utilize <code>unsigned long long</code> for size parameters to support large tensor dimensions.</li>
<li>Most kernels operate on raw float pointers for maximum flexibility and performance.</li>
<li>Kernel launch configurations (grid and block sizes) should be carefully managed to ensure optimal GPU utilization.</li>
</ul>
</dd></dl>
<dl class="section warning"><dt>Warning</dt><dd>These low-level CUDA kernel functions are intended for internal library implementation and framework extension. End-users building neural network models SHOULD NOT directly call these kernels. They are meant to be used exclusively by library developers contributing to the internal functionality of the nz framework.</dd></dl>
<dl class="section author"><dt>Author</dt><dd>Mgepahmge (<a href="https://github.com/Mgepahmge">https://github.com/Mgepahmge</a>)</dd></dl>
<dl class="section date"><dt>Date</dt><dd>2024/12/07 </dd></dl>

<p class="definition">Definition in file <a class="el" href="_operation_kernels_8cuh_source.html">OperationKernels.cuh</a>.</p>
</div></div><!-- contents -->
<!-- start footer part -->
<hr class="footer"/><address class="footer"><small>
Generated by&#160;<a href="https://www.doxygen.org/index.html"><img class="footer" src="doxygen.svg" width="104" height="31" alt="doxygen"/></a> 1.12.0
</small></address>
</div><!-- doc-content -->
</body>
</html>
